Monitoring computed tomography (CT) scan image

ABSTRACT

Disclosed is a system and a method for monitoring a CT scan image. A CT scan image may be resampled into a plurality of slices using a bilinear interpolation. A region of interest may be identified on each slice using an image processing technique. The region of interest may be masked on each slice using deep learning. Subsequently, a nodule may be detected as the region of interest using the deep learning. Further, a plurality of characteristics associated with the nodule may be identified. Furthermore, an emphysema may be detected in the region of interest on each slice. A malignancy risk score for the patient may be computed. A progress of the nodule may be monitored across subsequent CT scan images. Finally, a report of the patient may be generated.

RELATED APPLICATIONS

This application claims priority to Indian Patent Application No.202121045730 filed Oct. 7, 2021 in India.

TECHNICAL FIELD

The present subject matter described herein, in general, relates to asystem and a method for monitoring a Computed Tomography (CT) scanimage. More particularly, to monitoring a CT scan image using deeplearning.

BACKGROUND

Typically, medical imaging techniques such as Computed Tomography (CT)scans and E-radiations (X-ray) scans are widely used by a healthpractitioner to detect lung cancers. It must be noted that an earlydetection of cancerous nodule is really important. Generally, the healthpractitioner suggests a patient to go for the CT scans when he/shediagnose a presence of nodule in the chest of the patient. Further, thehealth practitioner analyses the CT scans and manually identify nodules.However, the manual detection is a time consuming and a cumbersome task.At times, the health practitioner may misjudge the nodules. Also, it mayhappen that the nodules, which are small in size, are missed by thehealth practitioner.

SUMMARY

Before the present system(s) and method(s), are described, it is to beunderstood that this application is not limited to the particularsystem(s), and methodologies described, as there can be multiplepossible embodiments which are not expressly illustrated in the presentdisclosures. It is also to be understood that the terminology used inthe description is for the purpose of describing the particularimplementations or versions or embodiments only and is not intended tolimit the scope of the present application. This summary is provided tointroduce aspects related to a system and a method for monitoring aComputed Tomography (CT) scan image. This summary is not intended toidentify essential features of the claimed subject matter nor is itintended for use in determining or limiting the scope of the claimedsubject matter.

In one implementation, a method for monitoring a Computed Tomography(CT) scan image is disclosed. Initially, a CT scan image of a patientmay be received. Further, a gaussian smoothing method may be applied onthe CT scan image to counteract noise. Subsequently, the CT scan imagemay be resampled into a plurality of slices. In one aspect, the CT scanimage may resample using a bilinear interpolation. Upon resampling, aregion of interest on each slice may be identified. In one aspect, theregion of interest may be identified using an image processingtechnique. Further, the region of interest on each slice may be masked.In one aspect, the region of interest may be masked by removing black orair areas and fatty tissues around the region of interest using deeplearning. Furthermore, a nodule may be detected as the region ofinterest using deep learning. Upon detection of the nodule, a pluralityof characteristics associated with the nodule may be determined usingthe image processing technique. In one aspect, the plurality ofcharacteristics may comprise a diameter, a calcification, a lobulation,a spiculation, a volume and a texture. Subsequently, an emphysema may bedetected in the region of interest on each slice using deep learning.

Upon detection, a malignancy risk score for the patient may be computedbased on the plurality of characteristics and trained data model. In oneaspect, the trained data model may comprise historical data related todifferent diameter of nodules, different calcification of nodules,different lobulation of nodules, different spiculation of nodules,different volume of nodules, and different texture of nodules. Duringimplementation, a progress of the nodule may be monitored over apredefined time period across subsequent CT scan images. In one aspect,the progress of the nodule may be monitored based on the diameter, atotal volume of the nodule, and the malignancy risk score. Finally, areport of a patient may be generated upon monitoring the progress of thenodule. In one aspect, the report may comprise the nodule, theemphysema, the malignancy risk score, the progress of the nodule and afollow-up check with a health practitioner, thereby monitoring the CTscan image. In one aspect, the aforementioned method for monitoring theCT scan image may be performed by a processor using programmedinstructions stored in a memory.

In another implementation, a non-transitory computer readable mediumembodying a program executable in a computing device for monitoring aComputed Tomography (CT) scan image is disclosed. The program maycomprise a program code for receiving a CT scan image of a patient.Further, the program may comprise a program code for applying a gaussiansmoothing method on the CT scan image to counteract noise. Subsequently,the program may comprise a program code for resampling the CT scan imageinto a plurality of slices. In one aspect, the CT scan image mayresample using a bilinear interpolation. Upon resampling, the programmay comprise a program code for identifying a region of interest on eachslice. In one aspect, the region of interest may be identified using animage processing technique. Further, the program may comprise a programcode for masking the region of interest on each slice. In one aspect,the region of interest may be masked by removing black or air areas andfatty tissues around the region of interest using deep learning.Furthermore, the program may comprise a program code for detecting anodule as the region of interest using deep learning. Upon detection ofthe nodule, the program may comprise a program code for determining aplurality of characteristics associated with the nodule using the imageprocessing technique. In one aspect, the plurality of characteristicsmay comprise a diameter, a calcification, a lobulation, a spiculation, avolume and a texture. Subsequently, the program may comprise a programcode for detecting an emphysema in the region of interest on each sliceusing deep learning.

Upon detection, the program may comprise a program code for computing amalignancy risk score for the patient based on the plurality ofcharacteristics and trained data model. In one aspect, the trained datamodel may comprise historical data related to different diameter ofnodules, different calcification of nodules, different lobulation ofnodules, different spiculation of nodules, different volume of nodules,and different texture of nodules. During implementation, the program maycomprise a program code for monitoring a progress of the nodule over apredefined time period across subsequent CT scan images. In one aspect,the progress of the nodule may be monitored based on the diameter, atotal volume of the nodule, and the malignancy risk score. Finally, theprogram may comprise a program code for generating a report of a patientupon monitoring the progress of the nodule. In one aspect, the reportmay comprise the nodule, the emphysema, the malignancy risk score, theprogress of the nodule and a follow-up check with a health practitioner,thereby monitoring the CT scan image.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing detailed description of embodiments is better understoodwhen read in conjunction with the appended drawings. For the purpose ofillustrating of the present subject matter, an example of constructionof the present subject matter is provided as figures, however, theinvention is not limited to the specific method and system formonitoring a CT scan image disclosed in the document and the figures.

The present subject matter is described in detail with reference to theaccompanying figures. In the figures, the left-most digit(s) of areference number identifies the figure in which the reference numberfirst appears. The same numbers are used throughout the drawings torefer various features of the present subject matter.

FIG. 1 illustrates a network implementation of a system for monitoring aCT scan image, in accordance with an embodiment of the present subjectmatter.

FIG. 2 shows a structure of a nodule, in accordance with an embodimentof the present subject matter.

FIG. 3 shows a structure of an emphysema, in accordance with anembodiment of the present subject matter.

FIG. 4 illustrates a method for monitoring a CT scan image, inaccordance with an embodiment of the present subject matter.

The figures depict an embodiment of the present disclosure for purposesof illustration only. One skilled in the art will readily recognize fromthe following discussion that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles of the disclosure described herein.

DETAILED DESCRIPTION

Some embodiments of this disclosure, illustrating all its features, willnow be discussed in detail. The words “receiving,” “applying”“resampling,” “identifying,” “masking,” “determining,” “detecting,”“computing,” “monitoring,” “generating,” and other forms thereof, areintended to be open ended in that an item or items following any one ofthese words is not meant to be an exhaustive listing of such item oritems, or meant to be limited to only the listed item or items. It mustalso be noted that as used herein and in the appended claims, thesingular forms “a,” “an,” and “the” include plural references unless thecontext clearly dictates otherwise. Although any system and methodssimilar or equivalent to those described herein can be used in thepractice or testing of embodiments of the present disclosure, theexemplary, system and methods are now described.

The disclosed embodiments are merely examples of the disclosure, whichmay be embodied in various forms. Various modifications to theembodiment will be readily apparent to those skilled in the art and thegeneric principles herein may be applied to other embodiments. However,one of ordinary skill in the art will readily recognize that the presentdisclosure is not intended to be limited to the embodiments described,but is to be accorded the widest scope consistent with the principlesand features described herein.

The present subject matter discloses a system and a method formonitoring a CT scan image. Typically, a doctor has to manually identifynodules in the CT scan image. This is a cumbersome and a time-consumingtask. More importantly, the present invention discloses a costeffective, efficient, and an automatic process for monitoring the CTscan image. The present invention generates a real-time report based onmonitoring the CT scan image. Further, the present invention providesremote assessment of the CT scan image. This helps to provideconsultation to a patient remotely. Initially, the CT scan image of thepatient may be received. Further, a region of interest may beidentified. Furthermore, a nodule and an emphysema may be detected usingdeep learning. The nodule may be further monitored over a predefinedtime period. Finally, a report of the patient may be generated uponmonitoring the nodule.

While aspects of described system and method for monitoring a ComputingTomography (CT) scan image may be implemented in any number of differentcomputing systems, environments, and/or configurations, the embodimentsare described in the context of the following exemplary system.

Referring now to FIG. 1 , a network implementation 100 of a system 102for monitoring a Computed Tomography (CT) scan image is disclosed. Itmay be noted that one or more users may access the system 102 throughone or more user devices 104-2, 104-3 . . . 104-N, collectively referredto as user devices 104, hereinafter, or applications residing on theuser devices 104. In one aspect, the one or more users may comprise ahealth practitioner, a doctor, a lab assistant, a radiologist and thelike.

Although the present disclosure is explained considering that the system102 is implemented on a server, it may be understood that the system 102may be implemented in a variety of computing systems, such as a laptopcomputer, a desktop computer, a notebook, a workstation, a virtualenvironment, a mainframe computer, a server, a network server, acloud-based computing environment. It will be understood that the system102 may be accessed by multiple users through one or more user devices104-1, 104-2 . . . 104-N. In one implementation, the system 102 maycomprise the cloud-based computing environment in which the user mayoperate individual computing systems configured to execute remotelylocated applications. Examples of the user devices 104 may include, butare not limited to, a portable computer, a personal digital assistant, ahandheld device, and a workstation. The user devices 104 arecommunicatively coupled to the system 102 through a network 106.

In one implementation, the network 106 may be a wireless network, awired network, or a combination thereof. The network 106 can beimplemented as one of the different types of networks, such as intranet,local area network (LAN), wide area network (WAN), the internet, and thelike. The network 106 may either be a dedicated network or a sharednetwork. The shared network represents an association of the differenttypes of networks that use a variety of protocols, for example,Hypertext Transfer Protocol (HTTP), Transmission ControlProtocol/Internet Protocol (TCP/IP), Wireless Application Protocol(WAP), and the like, to communicate with one another. Further, thenetwork 106 may include a variety of network devices, including routers,bridges, servers, computing devices, storage devices, and the like.

In one embodiment, the system 102 may include at least one processor108, an input/output (I/O) interface 110, and a memory 112. The at leastone processor 108 may be implemented as one or more microprocessors,microcomputers, microcontrollers, digital signal processors, CentralProcessing Units (CPUs), state machines, logic circuitries, and/or anydevices that manipulate signals based on operational instructions. Amongother capabilities, the at least one processor 108 is configured tofetch and execute computer-readable instructions stored in the memory112.

The I/O interface 110 may include a variety of software and hardwareinterfaces, for example, a web interface, a graphical user interface,and the like. The I/O interface 110 may allow the system 102 to interactwith the user directly or through the client devices 104. Further, theI/O interface 110 may enable the system 102 to communicate with othercomputing devices, such as web servers and external data servers (notshown). The I/O interface 110 can facilitate multiple communicationswithin a wide variety of networks and protocol types, including wirednetworks, for example, LAN, cable, etc., and wireless networks, such asWLAN, cellular, or satellite. The I/O interface 110 may include one ormore ports for connecting a number of devices to one another or toanother server.

The memory 112 may include any computer-readable medium or computerprogram product known in the art including, for example, volatilememory, such as static random access memory (SRAM) and dynamic randomaccess memory (DRAM), and/or nonvolatile memory, such as read onlymemory (ROM), erasable programmable ROM, flash memories, hard disks,Solid State Disks (SSD), optical disks, and magnetic tapes. The memory112 may include routines, programs, objects, components, datastructures, etc., which perform particular tasks or implement particularabstract data types. The memory 112 may include programs or codedinstructions that supplement applications and functions of the system102. In one embodiment, the memory 112, amongst other things, serves asa repository for storing data processed, received, and generated by oneor more of the programs or the coded instructions.

As there are various challenges observed in the existing art, thechallenges necessitate the need to build the system 102 for monitoringthe CT scan image. At first, a user may use the user device 104 toaccess the system 102 via the I/O interface 110. The user may registerthe user devices 104 using the I/O interface 110 in order to use thesystem 102. In one aspect, the user may access the I/O interface 110 ofthe system 102. The detail functioning of the system 102 is describedbelow with the help of figures.

The present subject matter describes the system 102 for monitoring theCT scan image. The system 102 may monitor the CT scan image inreal-time. The CT scan image may be monitored using deep learning andimage processing technique. In order to monitor the CT scan image,initially, the system 102 may receive the CT scan image of a patient. Inone aspect, the CT scan image may be referred as a chest CT scan. The CTscan image may be a three-dimensional image.

In one aspect, the CT scan image may be a Non-contrast CT series withaxial cuts and soft reconstruction kernel which covers an entire Lung.The CT scan image may be one non-contrast CT series with consistentlyspaced axial slices. The CT scan image may comprise minimum of 40 axialslices in the series. The CT scan image may be available in a DigitalImaging and Communications in Medicine (DICOM) format. In one example, amaximum thickness of the CT scan image may be 6 mm.

In one embodiment, the system 102 may check if the CT scan image is nota Chest CT scan or if there is no plain axial series, then the CT scanimage may not be processed further. The system 102 may transmit aresponse to the user indicating that the uploaded series or the CT scanimage is not a valid.

In one embodiment, the system 102 may comprise a trained data model. Thetrained data model may comprise historical data related to previous CTscans of the patient, one or more CT scans associated with a set ofpatients and the like. In one example, the trained data model maycomprise dataset containing 120,000 Chest CTs used for training andinternally validating algorithms. The dataset may be referred to as‘development dataset’. The development dataset may be divided intotraining dataset and internal validation dataset using a 4:1 split. Theresultant validation dataset (20% of the entire data) may be used toestimate the performance of the trained data model and forhyper-parameter tuning. The splitting may be based on a patient identity(ID) eliminating any spillage between the training and the validationsplits.

In one aspect, the dataset in the trained data model may be large andvalid that results into multiple advantages. The dataset may comprise anadequate number of scans for all target abnormalities, allowing thedevelopment of accurate algorithm. An adequate number of control scanswith various non-target abnormalities and normal variations may belikely to be present in the dataset. It may reduce the chances thatthese occurrences will negatively impact performance when the algorithmis deployed in the real world on previously unseen data. The selectionof a large number of sources for the training data, rather than a largeamount of data from a single site, may be advantageous because it allowsthe algorithms to be trained on the CT scan images from a wide varietyof device manufacturers and CT protocols, without explicitly specifying.

In one embodiment, the system 102 may automate the checking of the DICOMfor age, body part, contrast/non-contrast, slice thickness, view, andkernel. The system 102 may use a separate module called a seriesclassifier which is described in detail in the preprocessing section.Further, the system 102 may check presence of a corresponding radiologyreport i.e., a ground truth, by matching the patient IDs. If no reportis found, the CT scan image may be excluded. Subsequently, the system102 may automate the checking of radiology report for the age. Thesystem 102 may identify a number of cases which are labelled as CHEST inthe DICOM attribute but are not actually Chest CT scan images, such CTscan images may be identified using the trained series classifier andnot used in training or testing (these can be considered outliers).

In one embodiment, only requirements for training the system 102 may bethe presence of the DICOM data and a text report. Once theserequirements are met, the concept of missing values or missing data maynot apply as it does for other machine learning algorithms. There may beno other exclusions from the training dataset.

In order to eliminate the possibility of data/label leakage, thetraining and the validation split may be performed using a randomizationprocedure at the patient level rather than at the Study level. The ChestCT scan image studies from an (de-identified) individual may be placedeither in the training set or the testing set, but not both. There maybe no train-test contamination as the sources of data that were used fortrain data are completely different to those used for collecting testdata. There may be no validation data contamination as the training andthe validation split is done based on the hash of a unique identifier.The input data is an image i.e., the CT scan image whereas the targetvariable is a binary output i.e., information related to the presence ofa particular abnormality is not encoded in the actual image. In oneexample, an abnormal Chest CT scan image may not have any explicitinformation that it is abnormal apart from features in the image data.

In one aspect, an automated natural language processing based labelingapproach may be chosen as the primary method for generating the groundtruth. Additionally, a number of pixel-level hand annotations may beused to either further improve accuracy or to provide input to asegmentation algorithm. It may be noted that an intended use of thesystem 102 is to aid in the interpretation of the Chest CT images,therefore the labeling method that largely depends on the radiologyreports, which are the ground truth for these images, is appropriate.

Each Chest CT scan image in the training dataset may have a singlecorresponding the ground truth, generated during the normal course ofclinical care. The ground truth includes at least the radiology reportand a biopsy report where available. The ground truth is used fortraining the algorithm. The radiologist reports may not be generatedspecifically for the purpose of training the system 102 but may beobtained retrospectively from anonymized clinical records. The source ofthe reports may be exactly the same as the source of the CT scan images.In one example, the qualifications of the radiologists who generatedthese reports may be one of—MD, Radiology: Doctor of Medicine inRadiology, DNB, Radiology: Diplomate of National Board, Radiology, andDMRD: Diploma in Medical Radio Diagnosis.

In one aspect, the report may be in a free text format, and a customNatural Language Processing (NLP) algorithm may be used to extractlabels corresponding to each of the abnormal findings (indications). Thelabels may be served as the ground truth. The NLP algorithm may usewell-established systems developed to manage typographic errors, detectnegations and identify synonyms.

The Natural Language Processing (NLP) algorithms may be developed basedon rules/dictionaries, trained with machine learning techniques, or acombination of the two approaches. Rule based NLP algorithm may use alist of manually created rules to parse the unorganized content andstructure it. Machine Learning (ML) based NLP algorithm, on the otherhand, may automatically generate the rules when trained on a largeannotated dataset. The rule-based NLP algorithm may be chosen over amachine-learning based NLP algorithm for the purpose of labelingradiology reports.

The rule-based NLP algorithm may have few advantages comprising clinicalknowledge can be manually incorporated into the rule-based NLPalgorithm. In order to capture this knowledge in the ML based algorithm,a huge amount of annotation may be required. Further, rules may bereadily added or modified to accommodate a new set of target findings inthe rule-based NLP algorithm.

Once the CT scan image is received, the system 102 may apply a gaussiansmoothing method on the CT scan image. The gaussian smoothing method maybe configured to counteract noise. In other words, the gaussiansmoothing method may reduce image noise and enhance a structure of theCT scan image. In one aspect, the gaussian smoothing may be applied in az dimension (i.e., longitudinal axis). In one example, a gaussian kernelused for the gaussian smoothing may have a sigma of 1 mm in the zdimension, and 0 in other dimensions. The gaussian smoothing may have anegligible effect on the CT scan image with thickness greater than 2 mmas the gaussian kernel decays by 95% at 2*sigma (=2 mm).

Subsequently, the system 102 may resample the CT scan image into aplurality of slices. The CT scan image may be resampled using a bilinearinterpolation. The bilinear interpolation may use the distance weightedaverage of the four nearest pixel values to estimate a new pixel value.In one aspect, the system 102 may resample the CT scan image so that itsslice thickness is around 2.5 mm. The system 102 may obtain a resamplingfactor by dividing 2.5 by the series' slice thickness and rounding theresult to an integer. The rounding may be used to ensure that there areno resampling artifacts.

Further, the system 102 may identify a region of interest on each slice.The region of interest may be identified using an image processingtechnique. In one aspect, the region of interest may indicate anabnormality on each slice. In one example, the region of interest maycorrespond to possible nodules on each slice.

In one embodiment, a small part of the CT scan image may be annotated ata pixel level which serve as the secondary labels to the trainingalgorithms. It may include the region of interest annotation (lung,diaphragm, mediastinum and ribs) as well as abnormality pixel-wiseannotation which are then used to derive the Region of Interest (ROI)level annotations. In one example, 5% of the Chest CT scan images may beduplicated, as a test for competency of the annotators. If there wasless than 75% concordance the CT scan image was re-annotated. Thesediscrepancies may be tracked as a way to continually test the accuracyof the annotations and the competency of the annotators.

Once the region of interest is identified, the system 102 may mask theregion of interest on each slice. In one aspect, the region of interestmay be masked by removing black or air areas and fatty tissues aroundthe region of interest. The region of interest may be masked using deeplearning.

In one embodiment, each slice, from the plurality of slices, may consistof a significant amount of black or air areas and fatty tissue aroundthe region of interest. These areas may not be necessary to evaluate theslice. Removing these areas may help to focus on the region of interest.The masking may help to improve performance of a detection algorithm.

In one aspect, the system 102 may compute a three-dimensional boundingbox around the masked region. The bounding box may be used to crop theslice. The mask may be computed using a separately trained 2D UNetsegmentation algorithm.

Subsequently, the system 102 may detect a nodule as the region ofinterest. The nodule may be detected using the deep learning. In oneaspect, the nodule may be present in multiple slices. A total number ofslices in which the nodule is present may be computed.

In one aspect, the system 102 may comprise a Se-ResNeXt50 model todetect the nodule. The Se-ResNeXt50 may be a modified version of ResneXt50 a popular neural network architecture which has 50 layers withincreased inter layer connectivity. The model may have 50 convolutionallayers, modified to take in regional information and softmax basedconfidences. Further, the system 102 may comprise U-Net and FPN. TheU-Net and FPN may be a popular segmentation architecture for biomedicalsegmentation. The U-Net and FPN may have five downsampling and fiveupsampling blocks of convolutional layers with skip connections.

Referring now to FIG. 2 , the structure of the nodule is shown, inaccordance with an embodiment of the present subject matter. In oneembodiment, the CT scan image 200 of the patient may be received.Further, the system 102 may detect the nodule 202. In one example, thenodule may be a rounded or irregular opacity, well or poorly defined,measuring up to 3 cm in the diameter.

In one embodiment, the system 102 may use the neural networkarchitecture for slice-wise inference. It is an FPN with SE-ResNext-50backbone with classification and segmentation heads. Weights of theConvolutional Neural Network (CNN) may be used to process each slice maybe ‘tied’ and thus share the same weights.

Slice level classification output may be pooled into scan level outputusing following operation as shown in equation 1.Pscan=Σi=0 to #slices wi*Psclicei  Equation 1

Wherein w_(i) may be softmax weights computed as shown in equation 2.wi=exp(Pslicei)/Σi=0 to #slicesexp(Psclicei)  Equation 2

In one aspect, essentially the system 102 may comprise a softer versionof max pooling used in CNNs. The operation may be referred as‘softmaxpooling’. The model architecture may comprise three outputs:scan-level probability, list of slice-level probabilities of presence ofnodules and a 3D segmentation mask of nodules.

Referring again to FIG. 1 , the system 102 may determine a plurality ofcharacteristics associated with the nodule. The plurality ofcharacteristics may be identified using the image processing technique.The plurality of characteristics may comprise a diameter, acalcification, a lobulation, a spiculation, a texture, a volume and thelike. In one aspect, the diameter may correspond to a size. Thecalcification may indicate an amount of calcium present. The lobulationmay indicate a location. The spiculation may indicate a border. In oneembodiment, the system 102 may use a Convolution Neural Network (CNN)module to determine the plurality of characteristics. In one example,the diameter of the nodule may be further used to determine a totalvolume of the nodule and an area covered by the nodule.

Further, the system 102 may detect an emphysema in the region ofinterest. The emphysema may be detected using the deep learning. In oneexample, the system 102 may comprise a detection module to detect thenodule and/or the emphysema. In one embodiment, the system 102 maycomprise SE-ResNet18 model to detect the emphysema on each slice. TheSe-Resnet-18 model may be a modified version of Resnet 18 architecturewith more inter layer connectivity to increase capacity to learn. Themodel may have same 18 layers as Resnet-18. The model may be slightlymodified to take in regional information and modified SoftMax basedconfidences.

Further, the system 102 may comprise U-Net and FPN. The U-Net and FPNmay be a popular segmentation architecture for biomedical segmentation.The U-Net and FPN may have five downsampling and five upsampling blocksof convolutional layers with skip connections.

Referring now to FIG. 3 , a structure of the emphysema is shown, inaccordance with an embodiment of the present subject matter. In oneembodiment, the CT scan image 300 may be received. Further, theemphysema 302 may be detected. The emphysema 302 may be a permanentlyenlarged airspaces distal to the terminal bronchiole with destruction ofalveolar walls. On the CT scan image, an appearance of the emphysema 302may consist of focal areas or regions of low attenuation, usuallywithout visible walls.

Referring again to FIG. 1 , the deep learning may be used to detect thenodule and/or the emphysema. The deep learning is a form of a machinelearning technique where the hypothesis set is composed of neuralnetworks i.e., Convolutional Neural Networks (CNN. Once the trained datamodel using the CNN is generated, the system may be locked, tested anddeployed.

In one embodiment, the CNN or ConvNet is a class of deep neuralnetworks, most commonly applied to analyzing visual images. Neuralnetworks may be composed of a large number of interconnected individualcomputational units, arranged in layers, each of which applies a learnedfunction to the input data. CNN may be used for image processing and maybe characterized by ‘convolution’ layers, which contain layers thatlearn the matrix operations required to efficiently process images.

In the embodiment, an output of the CNN may be a score between 0 and 1.When trained appropriately with a large dataset of images and thecorresponding ground truth, the CNN may output the probability that agiven image belongs to a certain class or contains a specificabnormality i.e., the nodule or the emphysema. The output of the noduledetection algorithm may be a bounding box that localizes the region ofinterest.

In the embodiment, the detection module may use a two-dimensionalclassification convolutional neural network trained to output theprobability or heatmap that an abnormality is present in each slice. Theslice level probability or heatmaps may be processed using a poolingoperation for both abnormalities along with an additional 3D CNN forlung nodules to reduce false positives.

In one aspect, the system 102 may resize each slice using bilinearinterpolation. Each slice may be resized based on the detection ofabnormality i.e., the nodule or the emphysema. In one example, for thenodule, the standard size may include pixel size (320, 320) slices forinitial network, 96×96×96 patches around median point of proposal for FPreduction. Further, for the emphysema, the standard size may include thepixel size (224, 224).

Upon detection, the system 102 may compute a malignancy risk score forthe patient. The malignancy risk score may be computed based on theplurality of characteristics and the trained data model. The malignancyrisk score may indicate if the nodule is malignant i.e., cancerous ornot.

In one aspect, the trained data model may comprise historical datarelated to different diameter of nodules, different calcification ofnodules, different lobulation of nodules, different spiculation ofnodules, different texture of nodules different volume of nodules, andthe like. In one example, the historical data may comprise previousclinical reports associated with a set of patients. The trained datamodel may be generated based on continuously learning data associatedwith the set of patients using the deep learning. The trained data modelmay enable an accurate analysis.

In one embodiment, the system 102 may compare the plurality ofcharacteristics with the historical data. In one aspect, the system 102may use a Convolutional Neural Network (CNN) model for the comparison.In one example, the diameter may be compared with the different diameterof nodules. The calcification may be compared with the differentcalcification of nodules. The lobulation may be compared with thedifferent lobulation of nodules. The spiculation may be compared withthe different spiculation of nodules. The texture may be compared withthe different texture of nodules. Based on the comparison, the system102 may computer the malignancy risk score for the patient. Themalignancy risk score may be computed in real-time.

In another embodiment, the system 102 may assign a weightage to eachcharacteristic. Further, the weightage and the plurality ofcharacteristics may be used to compute the malignancy risk score.

Subsequently, the system 102 may monitor a progress of the nodule over apredefined time period. The progress of the nodule may be monitoredacross subsequent CT scan images. The progress of the nodule may bemonitored based on the diameter, the total volume of the nodule, themalignancy risk score and the like. The progress of the nodule may bemonitored in real-time.

In one embodiment, the diameter may be compared with the differentdimeters of nodules. Further, the malignancy risk score may be comparedwith a predefined threshold score. The total volume of the nodule may becompared with a previous volume stored in the trained data model. In oneaspect, a Convolution Neural Network (CNN) may be used to perform thecomparison. Based on the comparison, the system 102 may determine theprogress of the nodule. In one example, the system 102 may predict theprogress of the module based on the malignancy risk score and thepredefined threshold score.

Further, the system 102 may generate a report of the patient uponmonitoring the progress of the nodule. The report may be generated inreal-time. The report may comprise the nodule, the emphysema, themalignancy risk score, the progress of the nodule and a follow-up checkwith a health practitioner. In one aspect, the health practitioner mayanalyze the report and provide consultation to the patient remotely. Inone aspect, the system 102 may notify the patient regarding thefollow-up check with a health practitioner. The follow-up check may benotified based on the malignancy risk score, the progress of the noduleand the like.

In one embodiment, the system 102 may comprise a false positivedetection module. The false detection module may determine falsepositive nodules on each slice. The false positive nodules may bedetermined using a 3-dimentional CNN model. The false positive nodulesmay be determined by comparing the nodules with the trained data model.In one embodiment, the system 102 may use the lobulation and thediameter to determine the false positive nodules.

In one aspect, for the nodule, the system 102 may calculate a medianpoint. Further, a 3D patch of dimension 96×96×96 with the median pointas the Centre may be extracted from the CT scan image (resampled suchthat 1 voxel=1 mm3). The patch may be processed using a 3D SE-ResNet18trained to classify patches into whether or not they contain nodule(s).In one embodiment, training data for the false positive reduction modulemay be synthesized by accumulating negative patches using randomsampling from normal Chest CT scan image s and positive patches usingpixel/ROI level annotations by experts.

In one embodiment, the system 102 may receive the CT scan image which isfurther processed through an API is first passed to a scan filtering andseries picking module. The scan filtering and series picking module mayexamine each series to determine if there is a plain i.e., non-contrastaxial series of the Chest CT scan image present. The plain axial seriesmay be further passed to an image reading module before sending the CTscan image to an abnormality-specific preprocessing modules and models.The scan filtering and series picking module may check the DICOM tags oneach series to determine if the series is CT Chest plain axial series.

In one embodiment, an Image reading module may take in the plain axialseries from the scan filtering and series picking module. Further, aseries of raw dicom files, each representing a slice, may be needed tobe read and aggregated into the three-dimensional image. Therefore,sorting of the slices may be important while reading dicom files. TheDICOM tag Image Position Patient may be used to sort the dicom fileswhen available and the software falls back to the dicom tag InstanceNumber when not available. In one aspect, the open-source medical imageprocessing library SimpleITK6 is used to implement the image readingmodule.

In one aspect, the dicom tag and the tag description may be provided inthe Table 1.

TABLE 1 Dicom tag and tag description DICOM tag used for seriesselection Tag description Study Description Institution-generateddescription or classification of the Study (component) performed. SeriesDescription Description of the Series Modality Type of equipment thatoriginally acquired the data used to create the images in this Series.Body Part Examined Text description of the part of the body examined.Image Orientation The direction cosines of the first row and the Patientfirst column with respect to the patient. Image Position The x, y, and zcoordinates of the upper left hand Patient corner (centre of the firstvoxel transmitted) of the image, in mm. Pixel Spacing Physical distancein the patient between the centre of each pixel, specified by a numericpair - adjacent row spacing (delimiter) adjacent column spacing in mm.Slice Location Relative position of the image plane expressed in mm.Slice Thickness Nominal slice thickness, in mm. Rows Number of rows inthe image. Columns Number of columns in the image. Image Type Imageidentification characteristics. See Section C.7.6.1.1.2 of dicomstandard for Defined Terms and further explanation. Convolution Kernel Alabel describing the convolution kernel or algorithm used to reconstructthe data Window Centre Window Centre for display. Window Width WindowWidth for display. Contrast Bolus Agent Contrast or bolus agent ContrastBolus Route Administration route of contrast agent Requested ContrastContrast agent requested for use in the Scheduled Agent Procedure StepManufacturer Manufacturer of the equipment that produced the CompositeInstances.

Referring now to FIG. 4 , a method 400 for monitoring a CT scan image isshown, in accordance with an embodiment of the present subject matter.The method 400 may be described in the general context of computerexecutable instructions. Generally, computer executable instructions caninclude routines, programs, objects, components, data structures,procedures, modules, functions, etc., that perform particular functionsor implement particular abstract data types.

The order in which the method 400 is described is not intended to beconstrued as a limitation, and any number of the described method blockscan be combined in any order to implement the method 400 or alternatemethods for monitoring the CT scan image. Additionally, individualblocks may be deleted from the method 400 without departing from thespirit and scope of the subject matter described herein. Furthermore,the method 400 for monitoring CT scan image can be implemented in anysuitable hardware, software, firmware, or combination thereof. However,for ease of explanation, in the embodiments described below, the method400 may be considered to be implemented in the above-described system102.

At block 402, a CT scan image of a patient may be received.

At block 404, a gaussian smoothing method may be applied on the CT scanimage to counteract noise.

At block 406, the CT scan image may be resampled into a plurality ofslices. In one aspect, the CT scan image may be resampled using abilinear interpolation.

At block 408, a region of interest may be identified. In one aspect, theregion of interest may be identified using an image processingtechnique.

At block 410, the region of interest may be masked on each slice. In oneaspect, the region of interest may be masked by removing black or airareas and fatty tissues around the region of interest using deeplearning.

At block 412, a nodule as the region of interest may be identified usingthe deep learning.

At block 414, a plurality of characteristics associated with the nodulemay be determined using the image processing technique. In one aspect,the plurality of characteristics may comprise a diameter, acalcification, a lobulation, a spiculation, a volume and a texture.

At block 416, an emphysema may be detected in the region of interest oneach slice using the deep learning.

At block 418, a malignancy risk score for the patient may be computedbased on the plurality of characteristics and trained data model. In oneaspect, the trained data model may comprise historical data related todifferent diameter of nodules, different calcification of nodules,different lobulation of nodules, different spiculation of nodules,different volume of nodules, and different texture of nodules.

At block 420, a progress of the nodule may be monitored over apredefined time period across subsequent CT scan images. In one aspect,the progress of the nodule may be monitored based on the diameter, atotal volume of the nodule and the malignancy risk score.

At block 422, a report of the patient may be generated upon monitoringthe progress of the nodule. In one aspect, the report may comprise thenodule, the emphysema, the malignancy risk score, the progress of thenodule and a follow-up check with a health practitioner, therebymonitoring the CT scan image.

Exemplary embodiments discussed above may provide certain advantages.Though not required to practice aspects of the disclosure, theseadvantages may include those provided by the following features.

Some embodiments of the system and the method enable monitoring a CTscan image using deep learning and image processing technique.

Some embodiments of the system and the method enable detecting a noduleand an emphysema using a Convolution Neural Network (CNN) model.

Some embodiments of the system and the method enable identifying falsepositive nodules using a 3-dimentional CNN model.

Some embodiments of the system and the method enable computing amalignancy risk score using a plurality of characteristics.

Some embodiments of the system and the method enable an increase inspeed of process for monitoring of the CT scan image.

Some embodiments of the system and the method enable an efficient and anaccurate process using the large dataset.

Although implementations for methods and system for monitoring a CT scanimage have been described in language specific to structural featuresand/or methods, it is to be understood that the appended claims are notnecessarily limited to the specific features or methods described.Rather, the specific features and methods are disclosed as examples ofimplementations for monitoring the CT scan image.

What is claimed is:
 1. A method for monitoring a Computed Tomography(CT) scan image, the method comprising: receiving, by a processor, a CTscan image of a patient; applying, by the processor, a gaussiansmoothing method on the CT scan image to counteract noise; resampling,by the processor, the CT scan image into a plurality of slices, whereinthe CT scan image is resampled using a bilinear interpolation;identifying, by the processor, a region of interest on each slice,wherein the region of interest is identified using an image processingtechnique; masking, by the processor, the region of interest on eachslice, wherein the region of interest is masked by removing black or airareas and fatty tissues around the region of interest using deeplearning; detecting, by the processor, a nodule as the region ofinterest using the deep learning, wherein the nodule is detected uponthe masking of the region of interest; determining, by the processor, aplurality of characteristics associated with the nodule using the imageprocessing technique, wherein the plurality of characteristics comprisea diameter, a calcification, a lobulation, a spiculation, a volume, anda texture; detecting, by the processor, an emphysema in the region ofinterest on each slice using the deep learning; computing, by theprocessor, a malignancy risk score for the patient in real-time based onthe plurality of characteristics and trained data model, wherein thetrained data model comprises historical data related to differentdiameter of nodules, different calcification of nodules, differentlobulation of nodules, different spiculation of nodules, differentvolume of nodules, and different texture of nodules, and wherein themalignancy risk score is dependent on a weightage of eachcharacteristic; monitoring, by the processor, a progress of the nodulein real-time over a predefined time period across subsequent CT scanimages, wherein the progress of the nodule is monitored based on thediameter, a total volume of the nodule and the malignancy risk score;and generating, by the processor, a report of the patient uponmonitoring the progress of the nodule, wherein the report comprises thenodule, the emphysema, the malignancy risk score, the progress of thenodule and a follow-up check with a health practitioner, therebymonitoring the CT scan image.
 2. The method as claimed in claim 1,further comprises resizing each slice using a bilinear interpolation. 3.The method as claimed in claim 1, further comprises predicting theprogress of the module based on the malignancy risk score and apredefined threshold score.
 4. The method as claimed in claim 1, furthercomprises determining false positive nodules on each slice using a3-dimentional CNN model, wherein the false positive nodules aredetermined by comparing the nodules with the trained data model.
 5. Asystem for monitoring a Computed Tomography (CT) scan image, the systemcomprising: a memory; and a processor coupled to the memory, wherein theprocessor is configured to execute instructions stored in the memory to:receive a CT scan image of a patient; apply a gaussian smoothing methodon the CT scan image to counteract noise; resample the CT scan imageinto a plurality of slices, wherein the CT scan image is resampled usinga bilinear interpolation; identify a region of interest on each slice,wherein the region of interest is identified using an image processingtechnique; mask the region of interest on each slice, wherein the regionof interest is masked by removing black or air areas and fatty tissuesaround the region of interest using deep learning; detect a nodule asthe region of interest using the deep learning, wherein the nodule isdetected upon the masking of the region of interest; determine aplurality of characteristics associated with the nodule using the imageprocessing technique, wherein the plurality of characteristics comprisea diameter, a calcification, a lobulation, a spiculation, a volume, anda texture; detect an emphysema in the region of interest on each sliceusing the deep learning; compute a malignancy risk score for the patientin real-time based on the plurality of characteristics and trained datamodel, wherein the trained data model comprises historical data relatedto different diameter of nodules, different calcification of nodules,different lobulation of nodules, different spiculation of nodules,different volume of nodules, and different texture of nodules, andwherein the malignancy risk score is dependent on a weightage of eachcharacteristic; monitor a progress of the nodule in real-time over apredefined time period across subsequent CT scan images, wherein theprogress of the nodule is monitored based on the diameter, a totalvolume of the nodule and the malignancy risk score; and generate areport of the patient upon monitoring the progress of the nodule,wherein the report comprises the nodule, the emphysema, the malignancyrisk score, the progress of the nodule and a follow-up check with ahealth practitioner, thereby monitoring the CT scan image.
 6. The systemas claimed in claim 5, further configured to resize each slice using abilinear interpolation.
 7. The system as claimed in claim 5, furtherconfigured to predict the progress of the module based on the malignancyrisk score and a predefined threshold score.
 8. The system as claimed inclaim 5, further configured to determine false positive nodules on eachslice using a 3-dimentional CNN model, wherein the false positivenodules are determined by comparing the nodules with the trained datamodel.
 9. A non-transitory computer program product having embodiedthereon a computer program for monitoring a Computed Tomography (CT)scan image, the computer program product storing instructions, theinstructions comprising instructions for: receiving a CT scan image of apatient; applying a gaussian smoothing method on the CT scan image tocounteract noise; resampling the CT scan image into a plurality ofslices, wherein the CT scan image is resampled using a bilinearinterpolation; identifying a region of interest on each slice, whereinthe region of interest is identified using an image processingtechnique; masking the region of interest on each slice, wherein theregion of interest is masked by removing black or air areas and fattytissues around the region of interest using deep learning; detecting anodule as the region of interest using the deep learning, wherein thenodule is detected upon the masking of the region of interest;determining a plurality of characteristics associated with the noduleusing the image processing technique, wherein the plurality ofcharacteristics comprise a diameter, a calcification, a lobulation, aspiculation and a texture; detecting an emphysema in the region ofinterest on each slice using the deep learning; computing a malignancyrisk score for the patient in real-time based on the plurality ofcharacteristics and trained data model, wherein the trained data modelcomprises historical data related to different diameter of nodules,different calcification of nodules, different lobulation of nodules,different spiculation of nodules, different volume of nodules, anddifferent texture of nodules, and wherein the malignancy risk score isdependent on a weightage of each characteristic; monitoring a progressof the nodule in real-time over a predefined time period acrosssubsequent CT scan images, wherein the progress of the nodule ismonitored based on the diameter, a total volume of the nodule and themalignancy risk score; and generating a report of the patient uponmonitoring the progress of the nodule, wherein the report comprises thenodule, the emphysema, the malignancy risk score, the progress of thenodule and a follow-up check with a health practitioner, therebymonitoring the CT scan image.